Social interaction for efficient agent learning from human reward

Abstract - Learning from rewards generated by a human trainer observing an agent in action has been proven to be a powerful method for teaching autonomous agents to perform challenging tasks, especially for those non-technical users. Since the efficacy of this approach depends critically on the reward the trainer provides, we consider how the interaction between the trainer and the agent should be designed so as to increase the efficiency of the training process. This article investigates the influence of the agent’s socio-competitive feedback on the human trainer’s training behavior and the agent’s learning. The results of our user study with 85 participants suggest that the agent’s passive socio-competitive feedback—showing performance and score of agents trained by trainers in a leaderboard—substantially increases the engagement of the participants in the game task and improves the agents’ performance, even though the participants do not directly play the game but instead train the agent to do so. Moreover, making this feedback active—sending the trainer her agent’s performance relative to others—further induces more participants to train agents longer and improves the agent’s learning. Our further analysis shows that agents trained by trainers affected by both the passive and active social feedback could obtain a higher performance under a score mechanism that could be optimized from the trainer’s perspective and the agent’s additional active social feedback can keep participants to further train agents to learn policies that can obtain a higher performance under such a score mechanism.Fundamental Research Funds for the Central Universities of China (Grant No. 841713015)China Postdoctoral Science Foundatio

Recovery of the Historical SN1957D in X-rays with Chandra

SN1957D, located in one of the spiral arms of M83, is one of the small number of extragalactic supernovae that has remained detectable at radio and optical wavelengths during the decades after its explosion. Here we report the first detection of SN1957D in X-rays, as part of a 729 ks observation of M83 with \chandra. The X-ray luminosity (0.3 - 8 keV) is 1.7 (+2.4,-0.3) 10**37 ergs/s. The spectrum is hard and highly self-absorbed compared to most sources in M83 and to other young supernova remnants, suggesting that the system is dominated at X-ray wavelengths by an energetic pulsar and its pulsar wind nebula. The high column density may be due to absorption within the SN ejecta. HST WFC3 images resolve the supernova remnant from the surrounding emission and the local star field. Photometry of stars around SN1957D, using WFC3 images, indicates an age of less than 10**7 years and a main sequence turnoff mass more than 17 solar masses. New spectra obtained with Gemini-South show that the optical spectrum continues to be dominated by broad [O III] emission lines, the signature of fast-moving SN ejecta. The width of the broad lines has remained about 2700 km/s (FWHM). The [O III] flux dropped precipitously between 1989 and 1991, but continued monitoring shows the flux has been almost constant since. In contrast, radio observations over the period 1990-2011 show a decline rate inf the flux proportional to t**-4, far steeper than the rate observed earlier, suggesting that the primary shock has overrun the edge of a pre-SN wind.Comment: 28 pages, including 3 tables and 7 figures, accepted for publication in Ap

Vegetation demographics in Earth System Models: A review of progress and priorities

Numerous current efforts seek to improve the representation of ecosystem ecology and vegetation demographic processes within Earth System Models (ESMs). These developments are widely viewed as an important step in developing greater realism in predictions of future ecosystem states and fluxes. Increased realism, however, leads to increased model complexity, with new features raising a suite of ecological questions that require empirical constraints. Here, we review the developments that permit the representation of plant demographics in ESMs, and identify issues raised by these developments that highlight important gaps in ecological understanding. These issues inevitably translate into uncertainty in model projections but also allow models to be applied to new processes and questions concerning the dynamics of real-world ecosystems. We argue that stronger and more innovative connections to data, across the range of scales considered, are required to address these gaps in understanding. The development of first-generation land surface models as a unifying framework for ecophysiological understanding stimulated much research into plant physiological traits and gas exchange. Constraining predictions at ecologically relevant spatial and temporal scales will require a similar investment of effort and intensified inter-disciplinary communication

The Birth of an Ultra-Luminous X-ray Source in M83

A previously undetected X-ray source (L_X<10**36 erg/s) in the strongly star-forming galaxy M83 entered an ultraluminous state between August 2009 and December 2010. It was first seen with Chandra on 23 December 2010 at L_X ~ 4 10**39 ergs/s, and has remained ultraluminous through our most recent observations in December 2011, with typical flux variation of a factor of two. The spectrum is well fitted by a combination of absorbed power-law and disk black-body models. While the relative contributions of the models varies with time, we have seen no evidence for a canonical state transition. The luminosity and spectral properties are consistent with accretion powered by a black hole with M_BH ~ 40-100 solar masses. In July 2011 we found a luminous, blue optical counterpart which had not been seen in deep HST observations obtained in August 2009. These optical observations suggest that the donor star is a low-mass star undergoing Roche-lobe overflow, and that the blue optical emission seen during the outburst is coming from an irradiated accretion disk. This source shows that ultraluminous X-ray sources (ULXs) with low-mass companions are an important component of the ULX population in star-forming galaxies, and provides further evidence that the blue optical counterparts of some ULXs need not indicate a young, high-mass companion, but rather that they may indicate X-ray reprocessing.Comment: 40 pages, 7 figures, accepted for publication in Ap

A newly recognized very young supernova remnant in M83

As part of a spectroscopic survey of supernova remnant candidates in M83 using the Gemini-South telescope and Gemini Multi-Object Spectrograph, we have discovered one object whose spectrum shows very broad lines at Hα, [O I] λλ6300, 6363, and [O III] λλ4959, 5007, similar to those from other objects classified as "late time supernovae". Although six historical supernovae have been observed in M83 since 1923, none were seen at the location of this object. Hubble Space Telescope (HST) Wide Field Camera 3 images show a nearly unresolved emission source, while Chandra and ATCA data reveal a bright X-ray source and nonthermal radio source at the position. Objects in other galaxies showing similar spectra are only decades post-supernova, which raises the possibility that the supernova that created this object occurred during the last century but was missed. Using photometry of nearby stars from the HST data, we suggest the precursor was at least 17 M ☉, and the presence of broad Hα in the spectrum makes a type II supernova likely. The supernova must predate the 1983 Very Large Array radio detection of the object. We suggest examination of archival images of M83 to search for evidence of the supernova event that gave rise to this object, and thus provide a precise age

Detection of Murine Leukemia Virus or Mouse DNA in Commercial RT-PCR Reagents and Human DNAs

The xenotropic murine leukemia virus (MLV)-related viruses (XMRV) have been reported in persons with prostate cancer, chronic fatigue syndrome, and less frequently in blood donors. Polytropic MLVs have also been described in persons with CFS and blood donors. However, many studies have failed to confirm these findings, raising the possibility of contamination as a source of the positive results. One PCR reagent, Platinum Taq polymerase (pol) has been reported to contain mouse DNA that produces false-positive MLV PCR results. We report here the finding of a large number of PCR reagents that have low levels of MLV sequences. We found that recombinant reverse-transcriptase (RT) enzymes from six companies derived from either MLV or avian myeloblastosis virus contained MLV pol DNA sequences but not gag or mouse DNA sequences. Sequence and phylogenetic analysis showed high relatedness to Moloney MLV, suggesting residual contamination with an RT-containing plasmid. In addition, we identified contamination with mouse DNA and a variety of MLV sequences in commercially available human DNAs from leukocytes, brain tissues, and cell lines. These results identify new sources of MLV contamination and highlight the importance of careful pre-screening of commercial specimens and diagnostic reagents to avoid false-positive MLV PCR results

Observing the Evolution of the Universe

How did the universe evolve? The fine angular scale (l>1000) temperature and polarization anisotropies in the CMB are a Rosetta stone for understanding the evolution of the universe. Through detailed measurements one may address everything from the physics of the birth of the universe to the history of star formation and the process by which galaxies formed. One may in addition track the evolution of the dark energy and discover the net neutrino mass. We are at the dawn of a new era in which hundreds of square degrees of sky can be mapped with arcminute resolution and sensitivities measured in microKelvin. Acquiring these data requires the use of special purpose telescopes such as the Atacama Cosmology Telescope (ACT), located in Chile, and the South Pole Telescope (SPT). These new telescopes are outfitted with a new generation of custom mm-wave kilo-pixel arrays. Additional instruments are in the planning stages.Comment: Science White Paper submitted to the US Astro2010 Decadal Survey. Full list of 177 author available at http://cmbpol.uchicago.ed

CMB-S4: Forecasting Constraints on Primordial Gravitational Waves

CMB-S4---the next-generation ground-based cosmic microwave background (CMB) experiment---is set to significantly advance the sensitivity of CMB measurements and enhance our understanding of the origin and evolution of the Universe, from the highest energies at the dawn of time through the growth of structure to the present day. Among the science cases pursued with CMB-S4, the quest for detecting primordial gravitational waves is a central driver of the experimental design. This work details the development of a forecasting framework that includes a power-spectrum-based semi-analytic projection tool, targeted explicitly towards optimizing constraints on the tensor-to-scalar ratio, $r$, in the presence of Galactic foregrounds and gravitational lensing of the CMB. This framework is unique in its direct use of information from the achieved performance of current Stage 2--3 CMB experiments to robustly forecast the science reach of upcoming CMB-polarization endeavors. The methodology allows for rapid iteration over experimental configurations and offers a flexible way to optimize the design of future experiments given a desired scientific goal. To form a closed-loop process, we couple this semi-analytic tool with map-based validation studies, which allow for the injection of additional complexity and verification of our forecasts with several independent analysis methods. We document multiple rounds of forecasts for CMB-S4 using this process and the resulting establishment of the current reference design of the primordial gravitational-wave component of the Stage-4 experiment, optimized to achieve our science goals of detecting primordial gravitational waves for $r > 0.003$ at greater than $5\sigma$, or, in the absence of a detection, of reaching an upper limit of $r < 0.001$ at $95\%$ CL.Comment: 24 pages, 8 figures, 9 tables, submitted to ApJ. arXiv admin note: text overlap with arXiv:1907.0447

The James Webb Space Telescope Mission

Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit.Comment: Accepted by PASP for the special issue on The James Webb Space Telescope Overview, 29 pages, 4 figure